Pandas Profiling Report

Dataset statistics

Number of variables	15
Number of observations	115064
Missing cells	127817
Missing cells (%)	7.4%
Duplicate rows	0
Duplicate rows (%)	0.0%
Total size in memory	13.3 MiB
Average record size in memory	121.0 B

Variable types

Numeric	12
Categorical	2
Boolean	1

Warnings

`MarkDown1` is highly correlated with `MarkDown4`	High correlation
`MarkDown4` is highly correlated with `MarkDown1`	High correlation
`Size` is highly correlated with `MarkDown1` and 1 other fields	High correlation
`MarkDown1` is highly correlated with `Size` and 2 other fields	High correlation
`MarkDown4` is highly correlated with `MarkDown1`	High correlation
`MarkDown5` is highly correlated with `Size` and 1 other fields	High correlation
`Unemployment` is highly correlated with `Store` and 3 other fields	High correlation
`MarkDown4` is highly correlated with `MarkDown1` and 1 other fields	High correlation
`Store` is highly correlated with `Unemployment` and 4 other fields	High correlation
`IsHoliday` is highly correlated with `MarkDown1` and 2 other fields	High correlation
`Size` is highly correlated with `Unemployment` and 3 other fields	High correlation
`MarkDown1` is highly correlated with `MarkDown4` and 2 other fields	High correlation
`MarkDown3` is highly correlated with `IsHoliday` and 1 other fields	High correlation
`Fuel_Price` is highly correlated with `Unemployment` and 4 other fields	High correlation
`MarkDown2` is highly correlated with `Date`	High correlation
`CPI` is highly correlated with `Unemployment` and 4 other fields	High correlation
`Temperature` is highly correlated with `Fuel_Price` and 2 other fields	High correlation
`Type` is highly correlated with `Store` and 1 other fields	High correlation
`Date` is highly correlated with `MarkDown4` and 6 other fields	High correlation
`Date` is highly correlated with `IsHoliday`	High correlation
`IsHoliday` is highly correlated with `Date`	High correlation
`MarkDown2` has 28627 (24.9%) missing values	Missing
`MarkDown3` has 9829 (8.5%) missing values	Missing
`MarkDown4` has 12888 (11.2%) missing values	Missing
`CPI` has 38162 (33.2%) missing values	Missing
`Unemployment` has 38162 (33.2%) missing values	Missing
`MarkDown5` is highly skewed (γ1 = 37.97681524)	Skewed
`Date` is uniformly distributed	Uniform

Reproduction

Analysis started	2022-06-05 15:27:11.020156
Analysis finished	2022-06-05 15:31:06.214742
Duration	3 minutes and 55.19 seconds
Software version	pandas-profiling v3.0.0
Download configuration	config.json

Store
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	45
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	22.23820656

Minimum	1
Maximum	45
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	1.8 MiB

Quantile statistics

Minimum	1
5-th percentile	3
Q1	11
median	22
Q3	33
95-th percentile	43
Maximum	45
Range	44
Interquartile range (IQR)	22

Descriptive statistics

Standard deviation	12.80992959
Coefficient of variation (CV)	0.5760324941
Kurtosis	-1.149779815
Mean	22.23820656
Median Absolute Deviation (MAD)	11
Skewness	0.07677311858
Sum	2558817
Variance	164.0942961
Monotonicity	Increasing

Histogram with fixed size bins (bins=45)

Value	Count	Frequency (%)
13	2836	2.5%
4	2803	2.4%
19	2799	2.4%
2	2797	2.4%
27	2791	2.4%
24	2790	2.4%
6	2788	2.4%
1	2783	2.4%
10	2782	2.4%
20	2774	2.4%
Other values (35)	87121	75.7%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1	2783	2.4%
2	2797	2.4%
3	2473	2.1%
4	2803	2.4%
5	2447	2.1%
6	2788	2.4%
7	2669	2.3%
8	2699	2.3%
9	2435	2.1%
10	2782	2.4%

Value	Count	Frequency (%)
45	2626	2.3%
44	2072	1.8%
43	1863	1.6%
42	1962	1.7%
41	2754	2.4%
40	2738	2.4%
39	2704	2.3%
38	1987	1.7%
37	2013	1.7%
36	1718	1.5%

Dept
Real number (ℝ_≥0)

Distinct	81
Distinct (%)	0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	44.33952409

Minimum	1
Maximum	99
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	1.8 MiB

Quantile statistics

Minimum	1
5-th percentile	4
Q1	18
median	37
Q3	74
95-th percentile	95
Maximum	99
Range	98
Interquartile range (IQR)	56

Descriptive statistics

Standard deviation	30.65641013
Coefficient of variation (CV)	0.6914014248
Kurtosis	-1.224242898
Mean	44.33952409
Median Absolute Deviation (MAD)	23
Skewness	0.3624198542
Sum	5101883
Variance	939.8154821
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
1	1755	1.5%
16	1755	1.5%
81	1755	1.5%
67	1755	1.5%
82	1755	1.5%
46	1755	1.5%
2	1755	1.5%
40	1755	1.5%
38	1755	1.5%
90	1755	1.5%
Other values (71)	97514	84.7%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
1	1755	1.5%
2	1755	1.5%
3	1755	1.5%
4	1755	1.5%
5	1738	1.5%
6	1577	1.4%
7	1755	1.5%
8	1755	1.5%
9	1754	1.5%
10	1755	1.5%

Value	Count	Frequency (%)
99	613	0.5%
98	1632	1.4%
97	1716	1.5%
96	1350	1.2%
95	1755	1.5%
94	1464	1.3%
93	1638	1.4%
92	1755	1.5%
91	1755	1.5%
90	1755	1.5%

Date
Categorical

HIGH CORRELATION
HIGH CORRELATION
UNIFORM

Distinct	39
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	1.8 MiB

2012-12-21	3002
2012-12-07	2989
2012-12-28	2988
2012-12-14	2986
2013-02-15	2984
Other values (34)	100115

Length

Max length	10
Median length	10
Mean length	10
Min length	10

Characters and Unicode

Total characters	1150640
Distinct characters	11
Distinct categories	2 ?
Distinct scripts	1 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	2012-11-02
2nd row	2012-11-02
3rd row	2012-11-02
4th row	2012-11-02
5th row	2012-11-02

Common Values

Value	Count	Frequency (%)
2012-12-21	3002	2.6%
2012-12-07	2989	2.6%
2012-12-28	2988	2.6%
2012-12-14	2986	2.6%
2013-02-15	2984	2.6%
2012-11-23	2976	2.6%
2012-11-09	2971	2.6%
2013-01-04	2964	2.6%
2013-02-08	2964	2.6%
2012-11-30	2962	2.6%
Other values (29)	85278	74.1%

Length

Histogram of lengths of the category

Value	Count	Frequency (%)
2012-12-21	3002	2.6%
2012-12-07	2989	2.6%
2012-12-28	2988	2.6%
2012-12-14	2986	2.6%
2013-02-15	2984	2.6%
2012-11-23	2976	2.6%
2012-11-09	2971	2.6%
2013-01-04	2964	2.6%
2013-02-08	2964	2.6%
2012-11-30	2962	2.6%
Other values (29)	85278	74.1%

Most occurring characters

Value	Count	Frequency (%)
0	244692	21.3%
-	230128	20.0%
1	224528	19.5%
2	215739	18.7%
3	114767	10.0%
5	29470	2.6%
4	23610	2.1%
7	20617	1.8%
6	20542	1.8%
8	14749	1.3%

Most occurring categories

Value	Count	Frequency (%)
Decimal Number	920512	80.0%
Dash Punctuation	230128	20.0%

Most frequent character per category

Decimal Number

Value	Count	Frequency (%)
0	244692	26.6%
1	224528	24.4%
2	215739	23.4%
3	114767	12.5%
5	29470	3.2%
4	23610	2.6%
7	20617	2.2%
6	20542	2.2%
8	14749	1.6%
9	11798	1.3%

Dash Punctuation

Value	Count	Frequency (%)
-	230128	100.0%

Most occurring scripts

Value	Count	Frequency (%)
Common	1150640	100.0%

Most frequent character per script

Common

Value	Count	Frequency (%)
0	244692	21.3%
-	230128	20.0%
1	224528	19.5%
2	215739	18.7%
3	114767	10.0%
5	29470	2.6%
4	23610	2.1%
7	20617	1.8%
6	20542	1.8%
8	14749	1.3%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	1150640	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
0	244692	21.3%
-	230128	20.0%
1	224528	19.5%
2	215739	18.7%
3	114767	10.0%
5	29470	2.6%
4	23610	2.1%
7	20617	1.8%
6	20542	1.8%
8	14749	1.3%

IsHoliday
Boolean

HIGH CORRELATION
HIGH CORRELATION

Distinct	2
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	1011.3 KiB

False	106136
True	8928

Common Values
Chart

Value	Count	Frequency (%)
False	106136	92.2%
True	8928	7.8%

Type
Categorical

HIGH CORRELATION

Distinct	3
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Memory size	1.8 MiB

A	58713
B	44500
C	11851

Length

Max length	1
Median length	1
Mean length	1
Min length	1

Characters and Unicode

Total characters	115064
Distinct characters	3
Distinct categories	1 ?
Distinct scripts	1 ?
Distinct blocks	1 ?

The Unicode Standard assigns character properties to each code point, which can be used to analyse textual variables.

Unique

Unique	0 ?
Unique (%)	0.0%

Sample

1st row	A
2nd row	A
3rd row	A
4th row	A
5th row	A

Common Values

Value	Count	Frequency (%)
A	58713	51.0%
B	44500	38.7%
C	11851	10.3%

Length

Histogram of lengths of the category

Pie chart

Value	Count	Frequency (%)
a	58713	51.0%
b	44500	38.7%
c	11851	10.3%

Most occurring characters

Value	Count	Frequency (%)
A	58713	51.0%
B	44500	38.7%
C	11851	10.3%

Most occurring categories

Value	Count	Frequency (%)
Uppercase Letter	115064	100.0%

Most frequent character per category

Uppercase Letter

Value	Count	Frequency (%)
A	58713	51.0%
B	44500	38.7%
C	11851	10.3%

Most occurring scripts

Value	Count	Frequency (%)
Latin	115064	100.0%

Most frequent character per script

Latin

Value	Count	Frequency (%)
A	58713	51.0%
B	44500	38.7%
C	11851	10.3%

Most occurring blocks

Value	Count	Frequency (%)
ASCII	115064	100.0%

Most frequent character per block

ASCII

Value	Count	Frequency (%)
A	58713	51.0%
B	44500	38.7%
C	11851	10.3%

Size
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION

Distinct	40
Distinct (%)	< 0.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	136497.6889

Minimum	34875
Maximum	219622
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	1.8 MiB

Quantile statistics

Minimum	34875
5-th percentile	39690
Q1	93638
median	140167
Q3	202505
95-th percentile	206302
Maximum	219622
Range	184747
Interquartile range (IQR)	108867

Descriptive statistics

Standard deviation	61106.92644
Coefficient of variation (CV)	0.4476773704
Kurtosis	-1.214415882
Mean	136497.6889
Median Absolute Deviation (MAD)	62140
Skewness	-0.3219495188
Sum	1.570597008 × 10¹⁰
Variance	3734056459
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=40)

Value	Count	Frequency (%)
39910	5803	5.0%
39690	5702	5.0%
203819	5589	4.9%
219622	2836	2.5%
205863	2803	2.4%
202307	2797	2.4%
204184	2791	2.4%
202505	2788	2.4%
151315	2783	2.4%
126512	2782	2.4%
Other values (30)	78390	68.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
34875	2447	2.1%
37392	2473	2.1%
39690	5702	5.0%
39910	5803	5.0%
41062	1863	1.6%
42988	1954	1.7%
57197	2614	2.3%
70713	2669	2.3%
93188	2682	2.3%
93638	2554	2.2%

Value	Count	Frequency (%)
219622	2836	2.5%
207499	2756	2.4%
206302	2745	2.4%
205863	2803	2.4%
204184	2791	2.4%
203819	5589	4.9%
203750	2754	2.4%
203742	2774	2.4%
203007	2757	2.4%
202505	2788	2.4%

Temperature
Real number (ℝ)

HIGH CORRELATION

Distinct	1236
Distinct (%)	1.1%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	53.94180387

Minimum	-7.29
Maximum	101.95
Zeros	0
Zeros (%)	0.0%
Negative	208
Negative (%)	0.2%
Memory size	1.8 MiB

Quantile statistics

Minimum	-7.29
5-th percentile	23.98
Q1	39.82
median	54.47
Q3	67.35
95-th percentile	83.82
Maximum	101.95
Range	109.24
Interquartile range (IQR)	27.53

Descriptive statistics

Standard deviation	18.72415289
Coefficient of variation (CV)	0.3471176629
Kurtosis	-0.4959672256
Mean	53.94180387
Median Absolute Deviation (MAD)	13.79
Skewness	-0.07357032708
Sum	6206759.72
Variance	350.5939014
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
57.25	312	0.3%
70.18	309	0.3%
70.74	309	0.3%
38.95	272	0.2%
70.01	263	0.2%
57.87	262	0.2%
85	261	0.2%
52.38	260	0.2%
79.15	260	0.2%
58.66	259	0.2%
Other values (1226)	112297	97.6%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-7.29	69	0.1%
-6.61	69	0.1%
-6.08	70	0.1%
0.25	68	0.1%
2.32	71	0.1%
2.45	70	0.1%
4	70	0.1%
6.27	68	0.1%
8.55	69	0.1%
8.82	66	0.1%

Value	Count	Frequency (%)
101.95	187	0.2%
99.66	48	< 0.1%
95.51	45	< 0.1%
95.1	45	< 0.1%
94.1	45	< 0.1%
93.75	188	0.2%
93.62	44	< 0.1%
93.54	121	0.1%
93.17	47	< 0.1%
92.98	45	< 0.1%

Fuel_Price
Real number (ℝ_≥0)

HIGH CORRELATION

Distinct	297
Distinct (%)	0.3%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	3.581546322

Minimum	2.872
Maximum	4.125
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	1.8 MiB

Quantile statistics

Minimum	2.872
5-th percentile	3.161
Q1	3.431
median	3.606
Q3	3.766
95-th percentile	3.951
Maximum	4.125
Range	1.253
Interquartile range (IQR)	0.335

Descriptive statistics

Standard deviation	0.2394419304
Coefficient of variation (CV)	0.06685434414
Kurtosis	-0.1175964872
Mean	3.581546322
Median Absolute Deviation (MAD)	0.167
Skewness	-0.391281194
Sum	412107.046
Variance	0.05733243801
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
3.417	1853	1.6%
3.583	1851	1.6%
3.386	1793	1.6%
3.611	1374	1.2%
3.108	1201	1.0%
3.479	1169	1.0%
3.597	1071	0.9%
3.451	1043	0.9%
3.227	1040	0.9%
3.614	1028	0.9%
Other values (287)	101641	88.3%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
2.872	276	0.2%
2.889	276	0.2%
2.914	193	0.2%
2.927	194	0.2%
2.957	279	0.2%
2.982	195	0.2%
2.998	282	0.2%
3.029	195	0.2%
3.034	278	0.2%
3.066	197	0.2%

Value	Count	Frequency (%)
4.125	166	0.1%
4.109	189	0.2%
4.104	186	0.2%
4.099	355	0.3%
4.079	282	0.2%
4.062	351	0.3%
4.045	569	0.5%
4.031	358	0.3%
4.029	281	0.2%
4.016	358	0.3%

MarkDown1
Real number (ℝ)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION

Distinct	1752
Distinct (%)	1.5%
Missing	149
Missing (%)	0.1%
Infinite	0
Infinite (%)	0.0%
Mean	7689.216439

Minimum	-2781.45
Maximum	103184.98
Zeros	0
Zeros (%)	0.0%
Negative	207
Negative (%)	0.2%
Memory size	1.8 MiB

Quantile statistics

Minimum	-2781.45
5-th percentile	189.49
Q1	1966.46
median	4842.29
Q3	9439.14
95-th percentile	23140.88
Maximum	103184.98
Range	105966.43
Interquartile range (IQR)	7472.68

Descriptive statistics

Standard deviation	10698.76072
Coefficient of variation (CV)	1.391398044
Kurtosis	22.87146067
Mean	7689.216439
Median Absolute Deviation (MAD)	3402.05
Skewness	4.172741821
Sum	883606307.1
Variance	114463480.9
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
10755.57	74	0.1%
9753.88	74	0.1%
5692.66	74	0.1%
13613.52	74	0.1%
20297.6	74	0.1%
4655.55	74	0.1%
22673.11	74	0.1%
13357.31	74	0.1%
3365.61	73	0.1%
4642.37	73	0.1%
Other values (1742)	114177	99.2%
(Missing)	149	0.1%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-2781.45	50	< 0.1%
-772.21	43	< 0.1%
-563.9	70	0.1%
-16.93	44	< 0.1%
2.14	46	< 0.1%
2.5	62	0.1%
2.82	44	< 0.1%
4	49	< 0.1%
4.62	49	< 0.1%
5.76	49	< 0.1%

Value	Count	Frequency (%)
103184.98	72	0.1%
95102.5	71	0.1%
88750.34	66	0.1%
84139.36	72	0.1%
80498.65	71	0.1%
77017.24	68	0.1%
75522.86	70	0.1%
74777.22	65	0.1%
73407.49	73	0.1%
72937.29	73	0.1%

MarkDown2
Real number (ℝ)

HIGH CORRELATION
MISSING

Distinct	1257
Distinct (%)	1.5%
Missing	28627
Missing (%)	24.9%
Infinite	0
Infinite (%)	0.0%
Mean	3734.051729

Minimum	-35.74
Maximum	71074.17
Zeros	0
Zeros (%)	0.0%
Negative	412
Negative (%)	0.4%
Memory size	1.8 MiB

Quantile statistics

Minimum	-35.74
5-th percentile	6.14
Q1	180.35
median	742.59
Q3	2735.67
95-th percentile	22671.67
Maximum	71074.17
Range	71109.91
Interquartile range (IQR)	2555.32

Descriptive statistics

Standard deviation	8323.495014
Coefficient of variation (CV)	2.229078657
Kurtosis	15.88072536
Mean	3734.051729
Median Absolute Deviation (MAD)	667.49
Skewness	3.740647282
Sum	322760229.3
Variance	69280569.25
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
0.01	346	0.3%
0.03	340	0.3%
82.92	217	0.2%
11	214	0.2%
3	209	0.2%
4	191	0.2%
104.92	141	0.1%
1.49	138	0.1%
0.06	138	0.1%
7.5	137	0.1%
Other values (1247)	84366	73.3%
(Missing)	28627	24.9%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-35.74	63	0.1%
-15.45	71	0.1%
-7.76	65	0.1%
-3.27	69	0.1%
-0.05	73	0.1%
-0.01	71	0.1%
0.01	346	0.3%
0.02	135	0.1%
0.03	340	0.3%
0.04	72	0.1%

Value	Count	Frequency (%)
71074.17	72	0.1%
59362.3	72	0.1%
56549.69	73	0.1%
52850.71	74	0.1%
52304.87	73	0.1%
50636.71	72	0.1%
49026.3	73	0.1%
47440.51	71	0.1%
47382.72	71	0.1%
46315.84	70	0.1%

MarkDown3
Real number (ℝ)

HIGH CORRELATION
MISSING

Distinct	1421
Distinct (%)	1.4%
Missing	9829
Missing (%)	8.5%
Infinite	0
Infinite (%)	0.0%
Mean	2403.088666

Minimum	-179.26
Maximum	149483.31
Zeros	0
Zeros (%)	0.0%
Negative	589
Negative (%)	0.5%
Memory size	1.8 MiB

Quantile statistics

Minimum	-179.26
5-th percentile	1.18
Q1	15.1
median	78.26
Q3	272.58
95-th percentile	2361.57
Maximum	149483.31
Range	149662.57
Interquartile range (IQR)	257.48

Descriptive statistics

Standard deviation	13767.93931
Coefficient of variation (CV)	5.729268132
Kurtosis	54.09068129
Mean	2403.088666
Median Absolute Deviation (MAD)	73.28
Skewness	7.146137688
Sum	252889035.8
Variance	189556152.9
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
1.2	599	0.5%
1	498	0.4%
0.6	419	0.4%
0.8	348	0.3%
2	324	0.3%
0.4	278	0.2%
0.2	272	0.2%
3	271	0.2%
5	271	0.2%
0.1	269	0.2%
Other values (1411)	101686	88.4%
(Missing)	9829	8.5%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-179.26	62	0.1%
-89.1	66	0.1%
-44.54	67	0.1%
-23.97	72	0.1%
-17.44	69	0.1%
-14.29	63	0.1%
-2.58	67	0.1%
-0.86	71	0.1%
-0.73	52	< 0.1%
0.06	62	0.1%

Value	Count	Frequency (%)
149483.31	73	0.1%
146394.44	72	0.1%
139621.51	72	0.1%
130129.11	70	0.1%
115048.81	73	0.1%
112255.67	72	0.1%
109976.14	70	0.1%
105691.67	72	0.1%
105146.3	72	0.1%
97533.4	72	0.1%

MarkDown4
Real number (ℝ_≥0)

HIGH CORRELATION
HIGH CORRELATION
HIGH CORRELATION
MISSING

Distinct	1483
Distinct (%)	1.5%
Missing	12888
Missing (%)	11.2%
Infinite	0
Infinite (%)	0.0%
Mean	3356.219071

Minimum	0.22
Maximum	65344.64
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	1.8 MiB

Quantile statistics

Minimum	0.22
5-th percentile	16.96
Q1	155.46
median	840.94
Q3	3096.92
95-th percentile	14191.01
Maximum	65344.64
Range	65344.42
Interquartile range (IQR)	2941.46

Descriptive statistics

Standard deviation	7570.501545
Coefficient of variation (CV)	2.255663705
Kurtosis	25.45178529
Mean	3356.219071
Median Absolute Deviation (MAD)	796.5
Skewness	4.66858587
Sum	342925039.8
Variance	57312493.64
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
3	171	0.1%
0.63	154	0.1%
358.15	145	0.1%
55.46	142	0.1%
2.61	141	0.1%
3.97	138	0.1%
4.88	137	0.1%
27.44	136	0.1%
970.77	134	0.1%
1.92	120	0.1%
Other values (1473)	100758	87.6%
(Missing)	12888	11.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
0.22	56	< 0.1%
0.63	154	0.1%
0.66	46	< 0.1%
0.78	54	< 0.1%
1.26	43	< 0.1%
1.47	69	0.1%
1.56	52	< 0.1%
1.74	48	< 0.1%
1.89	55	< 0.1%
1.92	120	0.1%

Value	Count	Frequency (%)
65344.64	72	0.1%
63830.91	71	0.1%
63130.81	70	0.1%
60065.82	72	0.1%
56735.25	72	0.1%
56600.97	71	0.1%
52850.8	65	0.1%
51587.03	72	0.1%
49809.3	68	0.1%
49781.21	69	0.1%

MarkDown5
Real number (ℝ)

HIGH CORRELATION
SKEWED

Distinct	1754
Distinct (%)	1.5%
Missing	0
Missing (%)	0.0%
Infinite	0
Infinite (%)	0.0%
Mean	3922.681189

Minimum	-185.17
Maximum	771448.1
Zeros	0
Zeros (%)	0.0%
Negative	136
Negative (%)	0.1%
Memory size	1.8 MiB

Quantile statistics

Minimum	-185.17
5-th percentile	540.89
Q1	1309.3
median	2390.43
Q3	4227.27
95-th percentile	9316.71
Maximum	771448.1
Range	771633.27
Interquartile range (IQR)	2917.97

Descriptive statistics

Standard deviation	19445.15075
Coefficient of variation (CV)	4.957107093
Kurtosis	1494.90905
Mean	3922.681189
Median Absolute Deviation (MAD)	1288.67
Skewness	37.97681524
Sum	451359388.3
Variance	378113887.5
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
3113.78	137	0.1%
5449.98	74	0.1%
22677.91	74	0.1%
18831.34	74	0.1%
2105.14	74	0.1%
21807.99	74	0.1%
2167.73	74	0.1%
1947.25	74	0.1%
7968.28	74	0.1%
1993.5	73	0.1%
Other values (1744)	114262	99.3%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
-185.17	63	0.1%
-37.02	73	0.1%
40.98	44	< 0.1%
60.92	65	0.1%
114.25	51	< 0.1%
134.47	49	< 0.1%
142.75	66	0.1%
149.87	66	0.1%
157.48	65	0.1%
163.01	46	< 0.1%

Value	Count	Frequency (%)
771448.1	71	0.1%
45648.88	69	0.1%
45050.55	70	0.1%
43336.34	70	0.1%
35238.98	72	0.1%
31990.74	69	0.1%
23963.38	69	0.1%
23525.36	70	0.1%
22677.91	74	0.1%
22400.65	69	0.1%

CPI
Real number (ℝ_≥0)

HIGH CORRELATION
MISSING

Distinct	360
Distinct (%)	0.5%
Missing	38162
Missing (%)	33.2%
Infinite	0
Infinite (%)	0.0%
Mean	176.9613466

Minimum	131.2362258
Maximum	228.9764563
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	1.8 MiB

Quantile statistics

Minimum	131.2362258
5-th percentile	131.4784
Q1	138.4020333
median	192.3044449
Q3	223.2445318
95-th percentile	227.7847527
Maximum	228.9764563
Range	97.7402305
Interquartile range (IQR)	84.8424985

Descriptive statistics

Standard deviation	41.2399666
Coefficient of variation (CV)	0.2330450541
Kurtosis	-1.858841625
Mean	176.9613466
Median Absolute Deviation (MAD)	36.4254189
Skewness	0.07144841262
Sum	13608681.48
Variance	1700.734845
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
132.7160968	2080	1.8%
139.1226129	1664	1.4%
201.0705712	825	0.7%
224.8025314	783	0.7%
131.537	704	0.6%
132.2725714	703	0.6%
131.2793548	702	0.6%
131.642	702	0.6%
131.4784	701	0.6%
132.5914516	698	0.6%
Other values (350)	67340	58.5%
(Missing)	38162	33.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
131.2362258	695	0.6%
131.2793548	702	0.6%
131.3258	696	0.6%
131.3766667	695	0.6%
131.4275333	693	0.6%
131.4784	701	0.6%
131.537	704	0.6%
131.642	702	0.6%
131.747	695	0.6%
131.852	696	0.6%

Value	Count	Frequency (%)
228.9764563	186	0.2%
228.8892482	60	0.1%
228.8020401	60	0.1%
228.7796682	208	0.2%
228.7298638	401	0.3%
228.714832	62	0.1%
228.6926456	68	0.1%
228.6428882	134	0.1%
228.6276239	63	0.1%
228.605623	68	0.1%

Unemployment
Real number (ℝ_≥0)

HIGH CORRELATION
MISSING

Distinct	89
Distinct (%)	0.1%
Missing	38162
Missing (%)	33.2%
Infinite	0
Infinite (%)	0.0%
Mean	6.868733167

Minimum	3.684
Maximum	10.199
Zeros	0
Zeros (%)	0.0%
Negative	0
Negative (%)	0.0%
Memory size	1.8 MiB

Quantile statistics

Minimum	3.684
5-th percentile	3.932
Q1	5.771
median	6.806
Q3	8.036
95-th percentile	9.91
Maximum	10.199
Range	6.515
Interquartile range (IQR)	2.265

Descriptive statistics

Standard deviation	1.583427353
Coefficient of variation (CV)	0.2305268402
Kurtosis	-0.6093291493
Mean	6.868733167
Median Absolute Deviation (MAD)	1.186
Skewness	0.1414012336
Sum	528219.318
Variance	2.507242182
Monotonicity	Not monotonic

Histogram with fixed size bins (bins=50)

Value	Count	Frequency (%)
6.237	3377	2.9%
9.91	2454	2.1%
6.17	2336	2.0%
6.266	2147	1.9%
5.372	1871	1.6%
8.036	1823	1.6%
3.932	1808	1.6%
7.107	1808	1.6%
7.439	1805	1.6%
8.625	1783	1.5%
Other values (79)	55690	48.4%
(Missing)	38162	33.2%

Minimum 5 values
Maximum 5 values

Value	Count	Frequency (%)
3.684	556	0.5%
3.879	650	0.6%
3.896	288	0.3%
3.921	932	0.8%
3.932	1808	1.6%
4.145	1268	1.1%
4.872	210	0.2%
4.893	246	0.2%
4.954	573	0.5%
4.983	700	0.6%

Value	Count	Frequency (%)
10.199	1731	1.5%
9.91	2454	2.1%
9.874	751	0.7%
9.151	588	0.5%
8.951	847	0.7%
8.934	1529	1.3%
8.839	1063	0.9%
8.796	476	0.4%
8.78	257	0.2%
8.75	928	0.8%

Pearson's r

The Pearson's correlation coefficient (r) is a measure of linear correlation between two variables. It's value lies between -1 and +1, -1 indicating total negative linear correlation, 0 indicating no linear correlation and 1 indicating total positive linear correlation. Furthermore, r is invariant under separate changes in location and scale of the two variables, implying that for a linear function the angle to the x-axis does not affect r.

To calculate r for two variables X and Y, one divides the covariance of X and Y by the product of their standard deviations.

Spearman's ρ

The Spearman's rank correlation coefficient (ρ) is a measure of monotonic correlation between two variables, and is therefore better in catching nonlinear monotonic correlations than Pearson's r. It's value lies between -1 and +1, -1 indicating total negative monotonic correlation, 0 indicating no monotonic correlation and 1 indicating total positive monotonic correlation.

To calculate ρ for two variables X and Y, one divides the covariance of the rank variables of X and Y by the product of their standard deviations.

Kendall's τ

Similarly to Spearman's rank correlation coefficient, the Kendall rank correlation coefficient (τ) measures ordinal association between two variables. It's value lies between -1 and +1, -1 indicating total negative correlation, 0 indicating no correlation and 1 indicating total positive correlation.

To calculate τ for two variables X and Y, one determines the number of concordant and discordant pairs of observations. τ is given by the number of concordant pairs minus the discordant pairs divided by the total number of pairs.

Phik (φk)

Phik (φk) is a new and practical correlation coefficient that works consistently between categorical, ordinal and interval variables, captures non-linear dependency and reverts to the Pearson correlation coefficient in case of a bivariate normal input distribution. There is extensive documentation available here.

Cramér's V (φc)

Cramér's V is an association measure for nominal random variables. The coefficient ranges from 0 to 1, with 0 indicating independence and 1 indicating perfect association. The empirical estimators used for Cramér's V have been proved to be biased, even for large samples. We use a bias-corrected measure that has been proposed by Bergsma in 2013 that can be found here.

A simple visualization of nullity by column.

Nullity matrix is a data-dense display which lets you quickly visually pick out patterns in data completion.

The correlation heatmap measures nullity correlation: how strongly the presence or absence of one variable affects the presence of another.

The dendrogram allows you to more fully correlate variable completion, revealing trends deeper than the pairwise ones visible in the correlation heatmap.

First rows

	Store	Dept	Date	IsHoliday	Type	Size	Temperature	Fuel_Price	MarkDown1	MarkDown2	MarkDown3	MarkDown4	MarkDown5	CPI	Unemployment
0	1	1	2012-11-02	False	A	151315	55.32	3.386	6766.44	5147.7	50.82	3639.9	2737.42	223.462779	6.573
1	1	2	2012-11-02	False	A	151315	55.32	3.386	6766.44	5147.7	50.82	3639.9	2737.42	223.462779	6.573
2	1	3	2012-11-02	False	A	151315	55.32	3.386	6766.44	5147.7	50.82	3639.9	2737.42	223.462779	6.573
3	1	4	2012-11-02	False	A	151315	55.32	3.386	6766.44	5147.7	50.82	3639.9	2737.42	223.462779	6.573
4	1	5	2012-11-02	False	A	151315	55.32	3.386	6766.44	5147.7	50.82	3639.9	2737.42	223.462779	6.573
5	1	6	2012-11-02	False	A	151315	55.32	3.386	6766.44	5147.7	50.82	3639.9	2737.42	223.462779	6.573
6	1	7	2012-11-02	False	A	151315	55.32	3.386	6766.44	5147.7	50.82	3639.9	2737.42	223.462779	6.573
7	1	8	2012-11-02	False	A	151315	55.32	3.386	6766.44	5147.7	50.82	3639.9	2737.42	223.462779	6.573
8	1	9	2012-11-02	False	A	151315	55.32	3.386	6766.44	5147.7	50.82	3639.9	2737.42	223.462779	6.573
9	1	10	2012-11-02	False	A	151315	55.32	3.386	6766.44	5147.7	50.82	3639.9	2737.42	223.462779	6.573

Last rows

	Store	Dept	Date	IsHoliday	Type	Size	Temperature	Fuel_Price	MarkDown1	MarkDown2	MarkDown3	MarkDown4	MarkDown5	CPI	Unemployment
115054	45	85	2013-07-26	False	B	118221	76.06	3.804	212.02	851.73	2.06	10.88	1864.57	NaN	NaN
115055	45	87	2013-07-26	False	B	118221	76.06	3.804	212.02	851.73	2.06	10.88	1864.57	NaN	NaN
115056	45	90	2013-07-26	False	B	118221	76.06	3.804	212.02	851.73	2.06	10.88	1864.57	NaN	NaN
115057	45	91	2013-07-26	False	B	118221	76.06	3.804	212.02	851.73	2.06	10.88	1864.57	NaN	NaN
115058	45	92	2013-07-26	False	B	118221	76.06	3.804	212.02	851.73	2.06	10.88	1864.57	NaN	NaN
115059	45	93	2013-07-26	False	B	118221	76.06	3.804	212.02	851.73	2.06	10.88	1864.57	NaN	NaN
115060	45	94	2013-07-26	False	B	118221	76.06	3.804	212.02	851.73	2.06	10.88	1864.57	NaN	NaN
115061	45	95	2013-07-26	False	B	118221	76.06	3.804	212.02	851.73	2.06	10.88	1864.57	NaN	NaN
115062	45	97	2013-07-26	False	B	118221	76.06	3.804	212.02	851.73	2.06	10.88	1864.57	NaN	NaN
115063	45	98	2013-07-26	False	B	118221	76.06	3.804	212.02	851.73	2.06	10.88	1864.57	NaN	NaN

Overview

Variables

Common Values

Length

Most occurring characters

Most occurring categories

Most frequent character per category

Decimal Number

Dash Punctuation

Most occurring scripts

Most frequent character per script

Common

Most occurring blocks

Most frequent character per block

ASCII

Common Values

Length

Pie chart

Most occurring characters

Most occurring categories

Most frequent character per category

Uppercase Letter

Most occurring scripts

Most frequent character per script

Latin

Most occurring blocks

Most frequent character per block

ASCII

Interactions

Correlations

Pearson's r

Spearman's ρ

Kendall's τ

Phik (φk)

Cramér's V (φc)

Missing values

Sample

First rows

Last rows